What Is Apple’s Enhanced Visual Search and Should You Turn It Off?

Apple has always prided itself on keeping your data private, so it might come as a surprise that the Enhanced Visual Search feature in iOS 18 sends data about your photos to Apple’s servers by default, without ever having asked your permission to do so.

What Is Enhanced Visual Search?

Since iOS 15, you’ve been able to use a feature called Visual Look Up to identify objects such as plants and animals in the photos and videos on your iPhone. For example, if you open a photo you’ve taken of a plant in the Photos app, a symbol at the bottom of the screen indicates that Visual Look Up is available. Tap the icon, and you’ll see an identification of the plant in the image.

The same thing is possible with famous landmarks, animals, laundry tag symbols, and more. In iOS 18, Apple has taken this a step further, by introducing Enhanced Visual Search. As the name suggests, it’s Visual Look Up, only better. The key difference with Enhanced Visual Search is that, according to Apple, it can match your photos with “almost any landmark or point of interest.”

In order for this to work, however, some of your data needs to be sent from your iPhone to Apple’s servers so that a comparison can be made with the global index of landmarks. This has caused a reasonable amount of outrage among some people since Apple automatically opted every iPhone user into Enhanced Visual Search. It is possible to opt-out, but there are many people who see this as a major invasion of privacy.

While silently opting iPhone users into a feature that sends data from their device into the cloud isn’t ideal, the reality is that it’s not nearly as bad as it sounds. If you’ve seen headlines elsewhere, you may be forgiven for thinking that your iPhone is sending all of your photos to Apple for the company to examine at will. This isn’t the case, and Apple has tried to make the system as private as possible.

How Does Apple Keep Enhanced Visual Search Private?

The most important thing to know is that your iPhone is not sending all of your photos to Apple without your permission. It’s not even sending the relevant portions of your photos to Apple. Thanks to a technical paper Apple published on its website last year, we have a very detailed explanation of what’s actually going on, so strap in.

First, your iPhone examines your photos for anything that looks like a landmark. It then turns just that portion of the image into a vector embedding. This is a numerical representation of the image, not the image itself. In other words, it’s just a collection of numbers that describe the image of the Golden Gate Bridge, rather than the image itself.

These numbers are then encrypted using homomorphic encryption. This is a type of encryption that allows Apple’s servers to perform computations with the data while it’s still in its encrypted form. It can then send the result of that computation back, still encrypted, where it is finally decrypted on your device. This means the data from your iPhone is never decrypted on Apple’s servers.

To give a very simplistic analogy of how homomorphic encryption works, your iPhone could encrypt the number one as “apple” and the number two as “banana”. “Apple” and “banana” are sent to the cloud, and the server calculates that “apple” plus “banana” equals “eggplant.”

“Eggplant” is then sent back to your iPhone, and when decrypted gives the value of three. Apple’s servers have never seen values of one and two at any point; all calculations were performed using the encrypted values.

In the case of Enhanced Visual Search, the vector embedding describing the Golden Gate Bridge is encrypted and sent to Apple, where it is compared with the database of landmarks without ever being decrypted. The encrypted result is sent back and decrypted on your iPhone, which now knows that the landmark in your image is indeed the Golden Gate Bridge in San Francisco, and definitely not another bridge elsewhere.

To offer even more protection, the encrypted data from your iPhone is sent via an Oblivious HTTP (OHTTP) relay. This means that Apple can’t see where the data has come from, so there’s no way to link the data sent from your iPhone to your device. This is the same method used by Apple’s Private Relay to hide your IP address when you’re browsing in Safari.

In summary: your photos aren’t sent to Apple when Enhanced Visual Search is enabled. Only numerical data relating to portions of your photos that may contain landmarks are transmitted, and this data remains encrypted both during transit and when being processed. The chances of anyone, including Apple themselves, being able to see your iPhone photos when using Enhanced Visual Search are effectively zero.

How to Turn Off Enhanced Visual Search

Congratulations if you stayed awake through the section above. If you skipped straight to this one, all you need to know is that your images remain about as private as they can be when using Enhanced Visual Search. It’s far more likely for people to see your photos by taking your iPhone and guessing your passcode.

Even so, you may prefer that Apple doesn’t send any data from your iPhone to its servers just so that you can search for photos of the Taj Mahal. If so, you can turn off Enhanced Visual Search, which will stop your iPhone from sending this data to Apple.

Go to Settings > Apps > Photos. Scroll to the bottom of the screen, and toggle “Enhanced Visual Search” off. You can turn it back on again at any point you want.

There has been a lot of scaremongering about the fact that Enhanced Visual Search requires data to be sent to Apple to analyze, but the reality is that at no point are photos of you on vacation being uploaded to Apple and run through image recognition software. The reality is far more complex and boring, but also far more secure.

It’s not something you should be worrying about too much, especially if you already upload your photos to the cloud using services such as iCloud, Google Photos, or Dropbox. However, if you want to turn the feature off, the option is there.